716  HANDBOOK  FOR  EVALUATION  AND  LIFE  CVCLE  PLANNING  FOR  1/1 

SOFTWARE  VOLUME  3  REV.  .  (U)  ELECTRONIC  SVSTEMS  DIV 
HANSCOM  AFB  MA  M  FORSHEE  ET  AL.  01  FEB  83 
UNCLASSIFIED  ESD-TR-84-171-3  F/G  9/2  NL 


flflC  FILE  COP_Y  AD- A 142  716 


HANDBOOK  FOR  EVALUATION  AND  LIFE  CYCLE 
PLANNING  FOR  SOFTWARE 

Volume  III:  Reviews,  Audits  and  CPCI  Specifications 


Michael  Forshee,  Capt,  USAF 
Teresa  Fox-Daeke,  Capt,  USAF 
James  Dalrymple,  Capt,  USAF 
Dan  Papa,  Capt,  USAF 
Steve  Forest,  ILt,  USAF 
Guy  Ingram,  2Lt,  USAF 


1  February  1983 


APPROVED  FOR  PUBLIC  RELEASE 
DISTRIBUTION  UNLIMITED 


Prepared  for  m 

ELECTRONIC  SYSTEMS  DIVISION 

AIR  FORCE  SYSTEMS  COMMAND  \ 

DEPUTY  FOR  ACQUISITION  LOGISTICS 

AND  TECHNICAL  OPERATIONS 

HANSCOM  AIR  FORCE  BASE,  MASSACHUSETTS 


0173T 


S4  07  05  08ft 


LEGAL  NOTICE 


When  U.S.  Government  drawings,  specifications  or  other  data  are  used  for  any 
purpose  other  than  a  definitely  related  government  procurement  operation,  the 
government  thereby  Incurs  no  responsibility  nor  any  obligation  whatsoever;  and 
the  fact  that  the  government  may  have  formulated,  furnished,  or  in  any  way 
supplied  the  said  drawings,  specifications,  or  other  data  is  not  to  be  regarded 
by  implication  or  otherwise  as  in  any  manner  licensing  the  holder  or  any  other 
person  or  conveying  any  rights  or  permission  to  manufacture,  use,  or  sell  any 
patented  invention  that  may  in  any  way  be  related  thereto. 


OTHER  NOTICES 


Do  not  return  this  copy.  Retain  or  destroy. 


"This  technical  report  has  been  reviewed  and  is  approved  for  publication." 


REVTFWFD  3Y 


CHARLES  CARVALHO,  “JfT,  Capt ,  USAF  " 
Computer  Systems  Acquisitions  Manager 


WILLIAM  pTnELSOnT Capt,  USA/ 

Chief,  Computer  Resources  Engineering  Branch 


FOR  THE  COMMANDER 


Chief  Engineer  &  Assistant 
Deputy  for  Technical  Operations 


APPROVE  py 


JOSEFH  J.  FARINELLO,  GM-1 4 
Chief,  System  Quality  Division 
Deputy  for  Acquisition  Logistics 
and  Technical  Operations 


■  \  *. 

'  V' 


Unclassified 


SECURITY  CLASSIFICATION  of  THIS  page 


la  REPORT  SECURITY  CLASSIFICATION 

Unclassified 


2».  SECURITY  CLASSIFICATION  AUTHORITY 


REPORT  DOCUMENTATION  PAGE 


lb  RESTRICTIVE  MARKINGS 


3  DIST  Rl  BUTION/A  VAI  LA8I  LIT  Y  OF  REPORT 

Approved  for  public  release; 
Distribution  unlimited. 


5.  MONITORING  ORGANIZATION  REPORT  NUMBERlS) 


2b  DECLASSIFICATION, DOWNGRADING  SCHEOULE 


4  PERFORMING  ORGANIZATION  REPORT  NUMBERtSI 

AFCL-TR-84-1  71  (III) 


6a.  NAME  OF  PERFORMING  ORGANIZATION  6b.  OFFICE  SYMBOL  7a.  NAME  OF  MONITORING  ORGANIZATION 

(if  applicable ) 

Hq  Electronic  Systems  Division  ALEQ 


6c.  ADORESS  (Cit  y.  Stale  and  /IP  Code ) 

Hans  com  AFB 
Massachusetts  01731 


7b.  ADORESS  (City .  State  and  /IP  Code) 


8a.  NAME  OF  FUNDING/SPONSORING 
ORGANIZATION 


8c  AQORESS  (City.  State  and  ZIP  Code ) 


(If  applicable ) 


9.  PROCUREMENT  1 

NSTRUMENT  IDENTIFICATION  NUMBER 

10  SOURCE  OF  FUNDING  NOS. 

PROGRAM 

PROJECT 

TASK 

WORK  UNIT 

E  LEMENT  NO. 

| 

NO. 

NO.  1 

NO. 

12.  personal  AUTHOR(S)  Volume  III:  Reviews,  Audits  &  CPCI  Specifications  Steve  Forrsi.  li  t.  I'SAK 

Michael  Forshee.  Capt.  l.'SAF;  Teresa  Fox-Daeke,  Capt,  1SAF;  James  Dairy  in  plr,  Capt.  I  SAF;  Dan  I’apa.  (apt,  I'SAF;  (Juv  Ingram.  21.1.  I'S\F 


13b.  TIME  COVEREO 
FROM  _  TO  _ 


13a  TYPE  OF  REPORT 


16.  SUPPLEMENTARY  NOTATION 


14.  DATE  OF  REPORT  lYr.  Mo..  Day)  15.  PAGE  COUNT 

1983  February  1  44 


17. 

COSATI  COOES  | 

FfELD 

GROUP 

SUB.  GR.  1 

18.  SUBJECT  TERMS  ^Continue  on  reverse  if  neceuary  and  identify  by  block  number) 

SEE  REVERSE 


19L  ABSTRACT  (Continue  on  reverse  if  neceuary  and  identify  by  block  number ) 

^This  document  is  Volume  III  of  the  Handbook  for  Evaluation  and  Life  Cycle  Planning  for 
Software  (HELPS).  This  volume  provides  the  user  with  comprehensive  checklists  and 
guides  necessary  to  evaluate  a  contractor's  software  development  efforts.  These  include: 
documentation,  related  specifications,  support  analysis,  study  reports,  pertinent  design 
issues,  special  requirements,  interfacing,  and  other  relevant  action  items.  It  lists 
the  necessary  preparation  for  each  of  the  MIL-STD-1 521A  reviews  and  audits. 


20.  OlSTRIBUTION/AVAILASILITY  OF  ABSTRACT 

UN  CL  ASS  I F I E  O/UN  LI  Ml  TE  D  □  SAME  AS  RPT.  [3  OTIC  USERS  □ 


22».  NAME  OF  RESPONSIBLE  INDIVIDUAL 

Charles  Carvalho,  Jr,  Capt,  USAF 


21.  ABSTRACT  SECURITY  CLASSIFICATION 

Unclassified 


22b  TELEPHONE  NUMBER 
(Include  .Anea  Code ) 


DO  FORM  1473,  83  APR 


EDITION  OF  1  JAN  73  IS  OBSOLETE. 


Unclassified 

SECURITY  CLASSIFICATION  OF  THIS  PAGE 


Unclassified 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


Cont .  of  Block  18: 


SRR 

PCA 

CPPS 

LANGUAGES 

SQAP 

PROGRAMMING 

INTERFACE 

PACKAGING 


SDR 

FCA 

TEST 

DATA 

MAINTENANCE 

EXPANDABILITY 

TIMING 

SYSTEM 


PDR 

FQR 

REVIEWS 

MANAGEMENT 

CPCI 

PERFORMANCE 
SIZING 
B-5  SPEC 


CDR 

CPDS 

AUDITS 

CONFIGURATION 

REQUIREMENTS 

GFE 

STORAGE 
C-5  SPEC 


Unclassified 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


ACKNOWLEDGEMENTS 


This  book  was  developed  to  assist  ESD  computer  resource  personnel  in  the  performance 
of  their  daily  acquisition  related  activities.  It  contains  a  collection  of  available 
software  related  guidelines,  checklists,  and  standard  verbage  associated  with  AF  800 
series  software  and  computer  resources  life  cycle  activities.  The  activities  provided 
came  from  many  different  sources,  e.g.  guidebooks,  Computer  Resource  Acquisition 
Management  System  (CRAMS),  MITRE  studies,  information  exchanges  with  other  AFSC  product 
divisions,  and  most  importantlv,  from  experienced  software  acquisition  managers  here 
at  ESD. 


Accession  For 


NTIS  GRA4I 
DTIC  TAB 
Unannounced 
Justification. 


I 

□ 


By_ 

Distribution/ 


Availability  Codes 
(Avail  and/or 


HANDBOOK  FOR  EVALUATION  AND  LIFE  CYCLE  PLANNING  FOR  SOFTWARE 
VOLUME  III:  REVIEWS,  AUDITS,  AND  CPCI  SPECIFICATIONS 


TABLE  OF  CONTENTS 


PA 


Acknowledgement  s 
List  of  Figures 

Section  I:  Technical  Interchange  Meetings 

Section  2:  System  Requirements  Review/ System  Design  Review 

Section  3:  Preliminary  Design  Review 

Section  4:  Critical  Design  Review 

Section  5:  Physical  Configuration  Audit 

Section  6:  Functional  Configuration  Audit 

Section  7:  Formal  Qualification  Review 

Section  8:  Computer  Program  Development  Specification 

Section  9:  Computer  Program  Product  Specification 


ii 

vi 


1 

1 

1 

2 


List  of  References 


3 


HELPS,  Volume  III 

Reviews,  Audits,  and  CPCI  Specifications 


l  February  1 9 8 


Section  l  -  Tecnoicai  Interchange  Meetings  (TIMs) 

1-1.  Review  AFR  800-14,  VoL  II,  Para  4-10  for  compliance  reouirements. 

>.-2.  Review  items  for  discussion  at  TIM. 

1-3.  Review  the  following  significant  points  for  consideration: 

A.  Frequency  of  meetings 

B.  Representation  at  meetings 

C.  Actual  vs  Planning  Progress 

D.  Possible  misioterpr etation  of  computer  resource  design  or  functional 
requirements 

E.  Actual  vi  Planned  Expenditures  and  manpower  alignments 

F.  Resolution  of  problems 

1-4.  Prepare  aod  forward  agenda  to  the  contractors. 

1-5.  Meet  with  contractor  to  discuss  software  issues. • 

1-6.  Report  software  problems  to  program  office. 

1-7.  Track  aod  close  out  Government  software  Action  Items  generated  at  the 
TIMs. 

NUTE:  The  contractor  is  to  understand  that  the  PO  is  not  directing  him  to 
"get  into  the  minutes". 
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Section  2  -  System  Requirements  Review 
2-1.  Pre-Review  Preparation 

A.  Review  any  system  engineering  trade-studies  performed  to  support  the 
allocations  of  system  specification  functions  to  CPCIs. 

B.  Review  AFSCP  800-7,  Para  5. 3a. 

C.  Review  MIL-STD-1521A,  Appendix  A. 

D.  Review  ESD-TR-78- 1 17 ,  Software  Acquisition  Management  Guidebook: 
Reviews  and  Audits,  Section  3,  Paras  3. 1,  3. 1.1,  3.1.2,  3. 1.3. 

E.  Coordinate  the  SRR  agenda  (reference  MIL-STD-1521A,  Appendix  A,  Para 

10 . 3 )  • 

L.  Determine  if  the  interfaces  are  understood  and  reflected. 

2.  Review  initial  definitions  of  CPCIs. 

2-2-  SRR  Performance  Requirements 

A.  Participate  in  SSR  to  evaluate  the  developer's  progress  and  the 
direction  of  initial  effort. 

B.  Insure  requirement  of  representative  items  to  be  reviewed  are 
presented  during  review.  (MIL-STD-1521A,  Appendix  A) 

2-3-  Post-Review  Requirements 

A.  Review  SRR  minutes. 

B.  Monitor /resolve  action  items  pertinent  to  software. 

C.  Notify  the  contractor  (MIL-STD-1521A,  Para  4.2.4)  that  his  review 
performance  was  approved,  contingently  approved  or  disapproved. 
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I. 


System  Design  Review 


2-4.  Pre-Review  Preparation 

A.  Review  AFSCP  800-7  (Chapter  5)  4  AFR  800-14,  Vo l  II  (Paragraph  4-9b) . 

B.  Review  MIL-STD-1521A ,  Appendix  B. 

C.  Review  M1L-STD-L52LA,  4  -  L .  3  and  4.2  to  under* tand  contractor  and 
procuring  activity  requirements  for  supporting  reviews  and  audits. 

D.  Review  ESD-TR-77-254 ,  Section  2  to  establish  the  criteria  used  in 
defining  CPCIs. 

E.  Review  the  CPDP  and  MIL-STD-483,  Appendix  XVII. 

F.  Review  ESD-TR-78-I17 ,  Software  Acquisition  Management  Guidebook: 
Reviews  and  Audits,  Section  3i  Paras  3-2,  3*2.1,.  3*2.2,  3*2.3* 

G.  Coordinate  with  procurement  personnel  regarding  the  treatment  of 
competitive  information  for  each  program  prior  to  conducting  these  reviews. 

2-5.  SDR  Performance  Requirements 

A.  Attend  SDR  to  under* tand  and  review: 


1.  System  requirements. 

2.  Allocation  of  functions  to  CPCIs . 

3.  Contractor's  system  design  approach  using  personnel,  schedules 
and  hign  and  low  risk  rates. 

4.  Interface  requirements  from  CPCI  to  CPCI  and  CPCI  to  Cl. 

5.  The  computer  program  languages  to  be  used. 

B.  Identify  action  items  in  areas  where  presentation  is  deficient. 


2-6.  Post-Review  Activities 

A.  Review  SDR  minutes. 

B.  Monitor /resolve  action  items. 

C.  Inform  the  contractor  of  the  adequacy  of  SDR  by  notification  of 
either  approval,  contingent  approval,  or  disapproval  as  per  4.2.4  of 
MIL-STD-1521A. 
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Section  3  -  Preliminary  Design  Review 
3-1.  Pre-Review  Preparation 

A.  Insure  Keview  and  Autneotication  of  the  B-5  specifications, 
b.  Review  CPCI  Test  Plans. 

C.  Review  Checklist  for  a  Preliminary  Design  Review  for  a  CPCI, 
Figure:  III— 3— I. 


D.  Insure  that  areas  of  particular  Government  interest  are  forwarded  to 
the  contractor  enough  io  advance  to  allow  in-depth  preparation  time. 

E.  Review  PDR  agenda  (reference  MIL-STD-1521A,  Appendix  C). 

F.  Review  MIL-STD-1521A,  Section  4,  Para  4.2.4  and  Appendix  C  to 
under* tand  tne  options  available  to  the  PO  at  PDR  with  regards  to  approval, 
contingent  approval,  and  disapproval. 

3-2.  PDR  Performance  Requirements 

A.  Participate  in  the  PDR  at  the  contractor  facility. 

b.  Insure  subsystem  DT&E  Test  Plan  is  compatible  with  Section  4  of 
Development  Spec. 

C.  Insure  utility  and  support  software  requirements  are  defined  and 
schedu  Led. 

D.  Insure  contractor's  plan  for  controlling  changes  to  the  design  are 
established,  understood,  and  followed. 

E.  Insure  items  which  are  marked  N/A  or  TBD  are  explained  and  justified 
by  contractor. 

F.  Identify  action  items  where  pr esentatioo/design  is  deficient. 

3-3.  Post-Review  Requirements 

A.  Review  PDR  minutes. 

b.  Monitor /resolve  action  items. 

C.  Assure  necessary  ECP  action  is  initiated  for  changes  to  established 
baseline  as  required. 

D.  Inform  tne  contractor  (MIL-STD-1521A,  Para  4.2.4)  that  his  review 
performance  was  approved,  contingently  approved,  or  disapproved. 
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FIGURE  III-3-1  -  CHECKLIST  FOR  A  PRELIMINARY  DESIGN  REVIEW  FOR  A  CPCI 

Tne  following  list  is  based  on  the  requirements  of  MIL-STD-1521  (USAF), 
particularly  Appendix  C.  Specific  instructions  to  the  contractor  (in  the  SOW 
or  CDRL  backup  sheets)  that  modify,  clarify  or  supplement  the  provisions  of 
MIL-STD-1521A  (USAF)  may  significantly  alter  tne  applicability  of  the 
following  questions.  Such  instructions  must  be  reflected  correspondingly  in 
tne  cnecklist.  Three  areas  of  consideration  have  been  identified: 

•  Planning  and  preparation; 

•  Conducting  the  PDR ;  and 

•  Follow-up. 

Planning  and  Preparation 

Snould  the  review  be  a  collective  review  of  two  or  more  CPCIs  that  are 
highly  ioter-related? 

Has  tne  PDR  been  scheduled  so  as  to  allow  sufficient  time  for  meaningful 
analysis  and  design  and  for  adequate  preparation  by  contractor  personnel? 

Will  tne  development  specification  be  available  and  approved  prior  to  the 
PDR? 

Have  tne  responsible  (knowledgeable)  procuring  activity  personnel  (or 
qualified  alternates)  been  designated?  Have  the  appropriate  personnel 
from  other  participating  agencies  (including  contractors)  been  invited? 

Has  the  contractor  been  instructed  to  emphasize  certain  key  topics, 
problem  areas,  risk  areas  (e.g.,  interfaces,  man/machine  interaction), 
alternate  design  approaches,  trade-off  study  reports,  performance 
estimates,  simulation  results? 

Has  the  contractor  submitted  an  agenda  in  accordance  with  applicable 
standards  (MIL-STD-1521)  and  specific  instructions  (the  SOW)?  Has  an 
approved  agenda  been  distributed? 

Are  there  action  items  from  prior  or  related  CI/CPCI  reviews,  previously 
identified  design  issues  or  provisional  (e.g.,  "to  be  determined  (TBD)") 
entries  which  should  be  addressed? 

Have  the  procuring  activity  personnel  (and  all  others  from  the 
participating  agencies)  familiarized  themselves  with  the  subject 
documentation  (development  and  preliminary  product  specification),  related 
specifications,  supporting  analysis  aod  study  reports,  pertinent  design 
issues  and  relevant  action  items? 

Have  they  been  cautioned  about  giving  directions  to  the  contractor  during 
the  PDH? 

Have  the  appropriate  procuring  activity  personnel  coordinated  their  views 
with  respect  to  the  relevant  issues  or  action  items? 
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Is  the  procuring  activity  PDR  co-chairman  aware  of  his  responsibility  and 
authority  in  conducting  the  PDR? 

Conducting  the  PDR 

Opening  considerations: 

Is  everyone  familiar  with  the  agenda  and  indicated  topics? 

Are  the  appropriate  contractor  personnel  present  (including  the 
stenographer )? 

Have  the  joint  aod  distinct  responsibilities  and  authority  of  the 
procuring  activity  and  contractor  co-chairman  been  mutually  agreed  to,  and 
understood  by  the  attendees? 

Has  the  procuring  activity  co-cnairman  established  tne  ground  rules  for 
introducing  questions  and  discussion,  limiting  discussion,  entering 
records  into  the  minutes,  establishing  and  staffing  working  groups,  and 
setting  suspense  dates? 

Has  the  need  for  any  side  meetings  (e.g.,  working  groups)  been  previously 
identified?  Have  they  been  appropriately  scheduled  aod  staffed? 

Closing  considerations: 

Have  all  PDR  agenda  items  been  addressed? 

Have  working  group  conclusions  or  recommendations  been  reported  and 
recorded  in  the  minutes? 

Have  tne  previously  identified  action  items,  issues  aod  TBDs  in  the 
specifications  been  adequately  addressed? 

Have  new  action  items  been  identified,  recorded  aod  responsible  agencies 
assigoed  (including  suspense  dates,  whenever  feasible)? 

Have  tne  proceedings  been  completely  aod  accurately  summarized?  Have 
minutes  of  earlier  sessions  beeo  reviewed  aod  approved? 

Fol  low-up 

Have  minutes  been  prepared  and  submitted  in  accordance  with  the  CDRL 
requirements? 

Are  the  minutes  complete  and  accurate  from  the  viewpoint  of  the  procuring 
activity?  Are  all  action  items  identified? 

Have  approved  minutes  beeo  distributed  in  accordance  with  the  CDRL? 

Are  all  action  items  resolved  or  being  resolved  in  accordance  with  their 
suspense  dates  (if  assigned)? 
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Have  aLL  tne  resulting  action  items,  issues,  conclusions  and 
recommendations  which  are  related  to  other  (subsequent)  reviews/audits 
been  noted  for  action  or  consideration  by  the  appropriate  procuring 
activity  personnel? 


Has  the  procuring  activity  determined  tne  overall  adequacy  of  the  PDR,  or 
the  need  to  re-accomplish  some  (or  all)  of  the  PDR?  Has  the  contractor 
been  notified  if  needed? 


'  it*  v*  it*  v  >  .»  . 


HELPS,  Volume  III 

Reviews,  Audits,  ana  CPCI  Specifications 


l  FeDruary  1983 


Section  4  -  CriticaL  Design  Review 
4-1.  Pre-Review  Preparation. 

A.  Review  MIL-STD-1521A,  Appendix  D  for  CDR  compliance  requirements. 

B.  Prepare  for  CDR  by  reviewing  the  draft  C5  Specifications  for  those 
CPCIs  pertinent  to  the  CDR. 

C.  Review  ESD-TR-76-86 ,  Monitoring  and  Reporting  Software  Development 
Status,  CDR,  Pgs  26-30,  to  prepare  for  CDR. 

D.  Review  ESD-TR-77-327 ,  Software  Maintenance,  Critical  Design  Review, 
Section  2.2.2,  Pgs  21-22,  for  full  compliance  requirements  for  maintainable 
software. 

E.  Review  final  CPCI  Test  Plans. 

F.  Review  CDR  agenda,  send  notice  to  contractor  of  major  interest  areas. 
4-2.  CDR  Performance  Requirements. 

A.  Insure  contractor  nas  all  required  data  available  for  the  design 
review.  Check  ESD-TR-77-263,  Verification,  (reference  on  Pg  28),  for  an 
essential  data  list. 

B.  Duriog  tne  CDR  at  tne  contractor  facility,  determine  the  following 
through  detailed,  analytical  questioning: 

1)  Are  all  the  requirements  of  the  CPCI  Development  Specification 
being  implemented? 

2)  Is  tne  detailed  design  compatible  with  the  design  structure 
presented  at  the  PDR? 

3)  Is  tne  detail  sufficient  for  coding? 

C.  Determine  if  the  design  is  technically  acceptable  and  inform 
contractor  of  decision. 

4-3.  Post-Review  Requirements 

A.  Monitor/resolve  action  items. 
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Section  5  -  Physical  Configuration  Audit 
5.1.  Pre-Audit  Requirements 

A.  Review  Appendix  F  of  MIL-STD-I52IA  (USAF)  for  all  compliance 
requirements. 

B.  Review  ESD-TR-78-117 ,  Reviews  and  Audits,  Pgs  52-71,  for  formats  anc 
forms  used  in  conducting  a  PCA. 

C.  Review  ESD-TR-77-255 ,  Software  Quality  Assurance,  Para  2.4. 1.4.?, 

Pgs  50-51  for  all  QA  Requirements  to  be  met. 

D.  Review  MIL-STD-483*  Paras  3*9.2  and  120.7  for  PCA  compliance 
requirements. 

E.  Establish  an  agenda  for  PCA  and  coordinate  with  contractor. 

F.  Ensure  all  review  materials  are  available  to  the  audit  team.  The 
list  coosists  of: 

1.  CPCI  Development  (Part  I)  Specification 

2.  Final  draft  of  CPCI  Product  (Part  II)  Specification 

3-  Version  description  document. 

4.  Manuscript  copy  of  positional  handbook 

5.  Manuscript  of  computer  programming  manual 

6.  Manuscript  of  computer  program  user's  manual 

7.  FCA  minutes 

8.  Configuration  index 

9.  Configuration  status  report 

10.  CPCI  development  record 

IL.  Proposed  DD  Form  2 50  (including  shortage  list) 

12.  List  of  changes  made  to  CPCI  made  during  qualification  testing 

13.  Identification  information:  nomenclature,  CPCI  identifiers, 
specification  identification  number,  code  identification  number 
(H  4-1  number) 

NOTE:  All  of  the  above  are  found  in  ESD-TR-H7,  Pgs  38-39* 


s."  •- 
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A.  Participate  in  Pnysical  Configuration  Audits: 

1.  Review  code  listings  found  in  tne  C-5  Specification  to  source 
code  printouts  used  to  assemble  or  compile  tne  master  tape. 

2.  Verify  flowcharts  found  in  the  C-5  Specifications  against  the 
code  listings. 

3.  Verity  that  tne  CPCI  documentation  satisfies  its  respective  data 
item  description  requirements  and  that  user  documentation  has 
been  approved. 

9.  Verify  that  all  ECPs  nave  been  incorporated  on  the  appropriate 
documents  aod  that  accurate  configuration  status  accounting 
records  nave  been  maintained. 

5.  Verify  that  all  the  problems  and  deficiencies  identified  in  the 
design  review  and  audit  minutes  are  corrected. 

6.  Identify  all  CPCI  shortages  aod  make-up  schedules  on  the 
appropriate  DD  Form  250. 

5-3.  Post-Audit  Actions 


Office. 


A.  Coordinate  DD  Form  250  with  the  Government  Procuring  Contracting 


b.  Review  PCA  minutes  aod  inform  contractor  of  aoy  post-audit  action. 
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5 •  CPCI  Test  Reports  of  all  formal  tests  completed 
6.  PDR/CDR  Minutes 
7*  CM  Status  Reports 

8.  List  of  Qualification  Tests  yet  to  be  accomplished. 

D.  Review  ESD-TR-78-117,  Reviews  and  Audits  pgs  43-50  for  forms  and 
formats  used  in  conducting  a  FCA. 

E.  Review  ESD-TR-77-255 .  Software  Quality  Assurance  2. 4.1. *4. 2,  pgs  50-51 
for  the  QA  requirements. 

F.  Review  MIL-STD-483  3-9-1  and  120.6  for  compliance  requirements. 

G.  Establish  an  agenda  for  FCA  and  coordinate  with  the  contractor. 

6-2  FCA  Conduct 

A.  Participate  in  Functional  Configuration  Audit  at  the  contractor ' s 
facility.  Determine  deferred  requirements  yet  to  be  qualified  and  schedule 
their  verification  during  system  DT&E. 

B.  Review  the  test  documentation  and  analysis  to  ensure  that  the 
functional/performance  requirements  of  the  B-5  specification  have  been 
successfully  qualified. 

C.  Determine  functions  that  failed  to  qualify  during  PQT  and  FQT  to 
assess  impact  on  System  DT&E,  and  authorize  deviations /waiver s  as  appropriate, 
to  allow  testing  to  proceed. 

6-3  Post-Audit  Actions 

A.  Monitor /resoLve  FCA  actions  items. 

B.  Review  FCA  minutes  or  report. 
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Section  7  -  Formal  Qualification  Review 
7-1.  Pre-Review  Preparation 

A.  Review  MIL-STD-483 ,  Para  3. 9-4  anti  Para  120.8  for  compliance 
requirements. 

B.  Establish  an  agenda  for  FQR  (see  MIL-STD-1521A ,  Appendix  G)  and 
coordinate  with  the  contractor. 

C.  Ensure  tnat  all  review  materials  are  available  to  the  audit  team 
(including  appropriate  B-5  specifications,  FCA  and  PCA  minutes,  system  DT4E 
r  epor  t  s ) . 

7-2.  FQR  Conduct 

A.  Review  system  DT&E  results  to  verify  that  all  deferred  CPCI 
qualiticatoo  testing  has  been  successfully  accomplished. 

B.  Update  DD  Form  25 0  for  CPCI  shortages  (e.g.,  deviations  and  waivers) 
aod  coordinate  with  Principal  Contracting  Officer. 
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Sect  100  8  -  Computer  Program  Development  Specification 
8-1.  Review  MIL-STD-483  Appendix  VI. 

8-2.  Review  ESD-TR-78-139  An  Air  Force  Guide  to  the  Computer  Program 
Development  Specification. 

8-3.  Review  MIL-STD-1679 ( NAVY) ,  Weapon  Systems  Software  Development. 

8-4.  Review  the  Computer  Program  Development  Specification  Checklist,  Figure 
ILL -8-1. 


8-3.  Review  B-5  Spec  for  tecnnical  adequacy,  testability,  traceability, 
internal/external  interfaces,  feasibility  of  computer  program  design  factors, 
ano  the  design  requirements  and  constraints. 

8-6.  Insure  Section  4  of  the  spec  contains  a  verification  cross  reference 
matrix  which  indicates  the  method  used  to  test  each  performance  requirement 
stated  in  Section  3. 

8-7 •  Insure  B-5  requirements  are  traceable  to  the  System  or  Segment 
Specification. 

8-8.  Consolidate  comments  from  other  agencies  and  forward  to  contractor. 

8-9-  Meet  with  contractor  to  resolve  document  deficiencies. 

8-10.  Review  all  changes  to  the  document  as  a  result  of  Government  comments. 
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Figure  1 1 1-8- 1 ,  COMPUTER  PROGRAM  DEVELOPMENT  SPECIFICATION  CHECKLIST 

(CPCI  Part  I  Specification  I AW  Notice  2  to  MIL-STD-483) 

inis  checklist  is  Dased  primarily  on  tne  requirements  of  MIL-STD-483 
( U3AF ) ,  Appendix  VI;  and  Data  Item  Description  DI-E-31I9B,  Computer  Program 
Development  Specification.  Tne  reviewer  of  a  computer  program  development 
specification  must  be  aware  of  not  only  these  standards  but  also  any 
supplemental  or  cootrary  direction  for  the  preparation  of  the  development 
specifications  supplied  in  the  Statement  of  Work  (SOW),  in  the  Contract  Data 
Requirements  List  (CDRL)  or  in  CDRL  backup  sheets. 

Tne  checklist  questions  are  numbered  sequentially  aod  are  grouped  according 
to  tne  specification  paragraphs  to  which  they  apply.  Unless  otherwise 
indicated,  the  parenthetical  numbers  following  the  questions  refer  to  related 
paragraphs  of  MIL-STD-483  (USAF).  Backup  sheets  have  been  provided  at  the  end 
of  this  questionnaire  for  listing  exceptions.  Wherever  there  is  a  "no"  answer, 
tne  exceptions  should  be  listed  with  accompanying  explanations  on  the  backup 
sheets. 

TITLE  PAGE  Yes  No 

(1)  Is  there  a  title  page?  (60.4)  (  )  (  ) 

(2)  Does  the  title  page  cooform  to  Figure  1?  (60.4)  (  )  (  ) 

l .  SCOPE 

1 .  I  Identil ication 

(3)  Is  there  an  approved  identification,  nomenclature  aod  (  )  (  ) 

authorized  abbreviation  for  the  computer  program? 

(60.4.1) 

(4)  Does  the  specification’s  identification  paragraph  begin  (  )  (  ) 

with  the  prescribed  opening  phrase?  (60.4.1) 

1.2  Functional  Summary 

(5)  Is  there  a  summary  of  the  purpose  of  the  specification?  (  )  (  ) 

(60.4.1) 

(6)  Is  tnere  a  brief  description  of  the  overall  computer  (  )  (  ) 

program  by  major  functions?  (60.4.1) 

2.  APPLICABLE  DOCUMENTS 

(  )  (  ) 


(7)  Are  documents  properly  listed?  (60.4.2) 
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3.  REQUIREMENTS 

3*1  Computer  Program  Definition  Yes 

3.1.1  System  Capacities 

(8)  Have  all  capacity  requirements  been  described?  (  ) 

(60. 5. 3d. 1) 

3.1.2  Interface  Requirements 

(9)  Have  all  equipment  and  computer  programs  which  will  (  ) 

interface  with  the  CPCI  been  specified?  (60. 5. 3. 1.2) 

(10)  Have  all  of  the  physical  characteristics  of  interfacing  (  ) 

equipment  that  will  influence  the  design  of  this  CPCI  been 
specified  either  directly  or  by  reference  to  other 
specifications?  (60. 5-3. 1.2) 

(11)  Will  the  CPCI  be  written  in  a  compiler  or  assembly  (  ) 

language  that  can  be  translated  by  a  compiler  or  assembler 

tnat  is  part  of  this  CPCI?  (60.5.3.1.2) 

(12)  Has  each  compiler  or  assembler,  if  any,  that  will  (  ) 

translate  some  or  all  of  this  CPCI's  source  language  been 
specified  as  one  of  the  CPCI's  interfaces?  (60. 5. 3-1. 2) 

3. 1.2. 1  Interface  Block  Diagram 

(13)  Has  an  interface  block  diagram  or  an  equivalent  (  ) 

representation  been  included  that  shows  the  functional 
interrelationships  between  the  CPCI  and  each  of  the  other 
identified  computer  programs  aod  equipment  with  which  it  will 
interface?  (60.5.3.1.2.1) 

(14)  Does  this  interface  r epreseotation  adequately  depict  the  (  ) 

CPCI's  functional  relationships  with  each  computer  program 

or  equipment  item  with  which  the  CPCI  interfaces? 

(60. 5. 3d. 2.1) 

3. 1.2.2  Detailed  Interface  Definition 

(15)  Have  the  functional  relationships  between  the  CPCI  (  ) 

and  each  of  the  interfacing  equipment  items  and  computer 

programs  been  described?  (60.5.3.1.2.2) 

(16)  Have  the  formats  for  the  initiation,  maintenance  aod  (  ) 

termination  of  interaction  with  all  interactive  interfaces 

been  specified?  (60.5.3.1.2.2) 

(17)  Have  the  capacity  requirements  (e.g.,  the  maximum  (  ) 

overall  data  rates),  for  each  interface  been  specified 
quantitatively?  (60.5.3.1.2.2) 


No 


(  ) 


(  ) 

(  ) 


(  ) 

(  ) 


(  ) 


(  ) 


(  ) 

(  ) 

(  ) 
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Yes  No 

(L8)  Have  all  other  timing  requirements  imposed  by  all  (  )  (  ) 

interfacing  equipment  or  computer  programs  been  specified? 

(60-5.3- 1*2*2) 

(19)  Has  tne  protocol  for  tne  initiation,  maintenance  and  (  )  (  ) 

termination  of  interaction  with  all  interactive  interfaces 

been  specified?  ,60. 5*3*1 *2.2) 

(20)  Have  all  other  design  constraints  imposed  by  the  (  )  (  ) 

interfacing  equipment  and  computer  programs  on  the  design 

of  the  CPCI  been  identified?  (60.5*3*1*2.2) 

(21)  For  each  of  the  CPCI's  interfaces,  are  the  functional  (  )  (  ) 

descriptions  adequate  to  support  the  design  of  the  CPCI? 

(60.5*3*1*2.2) 

3.2  Detailed  Functional  Requirements 

(22)  Have  all  operational  functions  (e.g.,  radar  input  (  )  (  ) 

processing,  weapons  direction,  fragmentary  order  generation) 

to  be  performed  by  tne  CPCI  been  specified?  (60.5*3*2) 

(23)  Have  all  special  functions  (such  as  control,  test  (  )  (  ) 

simulation,  recovery)  derived  from  the  operational 

performance  requirements  and  interface  requirements  been 
specified?  (60.5*3*2) 

(2*4)  Is  there  a  separately  numbered  and  titled  subparagraph  (  )  (  ) 

(i.e.,  3.2.1,  3*2.2,  3*2.3,  etc.)  for  each  of  these  CPCI's 
operational  and  special  functions?  (60.5*3*2) 

(25)  Is  there  a  functional  block  diagram  or  equivalent  (  )  (  ) 

graphical  representation  of  the  CPCI?  (60.5*3*2) 

(26)  Does  the  functional  block  diagram  or  equivalent  (  )  (  ) 

graphical  representation  adequately  illustrate  the  functional 

operation  of  the  CPCI?  (60*5*3*2) 

(27)  Does  the  functional  block  diagram  or  equivalent  (  )  (  ) 

graphical  representation  adequately  illustrate  the 

relationships  between  the  CPCI's  functions?  (60.5.3*2) 

(28)  Does  the  functional  block  diagram  or  equivalent  (  )  (  ) 

graphical  representation  adequately  illustrate  the 

re lationships  between  the  CPCI's  functions  and  the  things 
with  which  it  must  interface  (e.g.,  the  functions  of  the  other 
identified  systems,  CPCIs,  and  interfacing  equipment)? 

(60.5*3*2) 
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Yes  No 

3.2. X  Fuoction  X.  NOTE:  Each  function  identified  in  basic  paragraph  3.2  of  the 
specification  is  further  specified  in  a  subparagraph  numbered  3.2.X,  where  X  ranges 
from  l  to  the  total  number  of  functions,  and  "Function  X"  is  replaced  by  the  name 
of  the  corresponding  function.  All  auestions  under  3.2.X  should  be  answered  for 
each  of  the  CPCI’s  functions. 

(29)  Considering  the  content  of  higher-level,  lower  level,  (  )  (  ) 

aod  related  paragraphs,  does  the  descriptive  and  introductory 

material  provided  in  this  paragraph  adequately  define  the 
fuoction  aod  its  relationships  to  other  functions? 

(60. 5- 3-2. U 

3 . 2 .  X . 1  Inputs 

(30)  Iodicate  which  of  the  following  kinds  of  information  have 
been  specified  for  each  input  to  the  function.  (60.5-3.2. 1. 1) 

Name  or  ID  of  Name  of  ID  of  Name  or  ID  of 

Input  1  _  Input  2  _  Input  3# _ 

Input  Information  Yes  No  Yes  No  Yes  No 

Source(s)  (  )  (  )  (  )  (  )  (  )  (  ) 

Data  Format  (  )  (  )  (  )  (  )  (  )  (  ) 

Units  of  Measure  (  )  (  )  (  )  (  )  (  )  (  ) 

Data  Item  Legal  (  )  (  )  (  )  (  )  (  )  (  ) 

Value  Ranges 

Accuracy  (  )  (  )  (  )  (  )  (  )  (  ) 

Requirements 

Precision  (  )  (  )  (  )  (  )  (  )  (  ) 

Requirements 

Input  to  VoLume  (  )  (  )  (  )  (  )  (  )  (  ) 

Input  Frequency  (  )  (  )  (  )  (  )  (  )  (  ) 

(3D  Are  the  actions  to  be  taken  in  the  event  of  erroneous  (  )  (  ) 

input  data  specified  explicitly  either  or  an  individual 
input  basis,  input  category  basis,  or  on  some  other  broader 
basis?  (60 .5 • 3*2. 1 . 1) 

(32)  Are  the  diagnostic  or  error  messages  to  be  generated  (  )  (  ) 

in  the  event  of  an  error  specified  explicitly?  (60 .5 . 3.2. 1 . 1 ) 


#  Repeat  this  table  on  backup  sheets  for  any  additional  inputs 
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Yes  .Vo 


(33)  Has  tne  intent  of  tne  fuoct.no's  operations  been  stated  (  )  (  ) 

briefly?  (60. 5. 3-2. 1 .2) 

(34)  Has  eacn  processing  requirement  of  tbe  function  been  (  )  (  ) 

specified  such  that  the  conditions  under  which  the 

subfunctions  are  performed  are  identified,  which  specific 
mathematical  or  logical  operations  to  be  performed  ar  *? 
identified,  and  the  inputs  to  be  processed  and  the  outputs 
to  be  generated  are  identified?  (60.5.3.2.1.2) 

(35)  Has  an  accurate  textual  or  mathematical  description  of  (  )  (  ) 

each  processing  requirement  been  included?  (60. 5. 3. 2. 1.2) 

(36)  Have  sequencing,  timing,  accuracy  requirements  and  all  (  )  (  ) 

other  constraints  upon  the  subfunction  been  specified? 

(60. 5. 3- 2. 1.2) 

3. 2.X. 3  Outputs 

(37)  Iodicate  which  of  the  following  has  been  specified  for  each 
output  generated  by  this  function.  (60.5.3.2.1.3) 


Name  or  ID  of 

Name  of  ID  of 

Name 

or  ID  of 

Output 

1 

Output  2 

Output  3#  _ 

Output  Information 

Yes 

No 

Yes  No 

Yes 

No 

Data  Content 
Specified 

(  ) 

(  ) 

(  )  (  ) 

(  ) 

(  ) 

Conditions 

Specified 

(  ) 

(  ) 

(  )  (  ) 

(  ) 

(  ) 

Destination 

Specified 

(  ) 

(  ) 

(  )  (  ) 

(  ) 

(  ) 

Format  Specified 

(  ) 

(  ) 

(  )  (  ) 

(  ) 

Yes 

(  ) 

No 

(38)  Have  the  units  of  measure,  accuracy/precision 
requirements,  and  frequency  of  output  information  been 
specified?  (60.5*3.2.1.3) 

(  ) 

(  ) 

Name  or  I D 

1  Of 

Name  or  ID  of 

Name  or  ID  of 

Name  or 

ID  of 

Output  l 

Output  2  _ 

Output  3 

Output 

4# 

Output  Yes 

No 

Yes  No 

Yes  No 

Yes 

No 

Units  of  (  )  ( 

) 

(  )  (  ) 

(  )  (  ) 

(  ) 

(  ) 

Measure 


t  Repeat  this  table  on  backup  sheets  for  any  additional  outputs. 
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Output 
(cont. ) 

Yes  No 

Yes 

No 

Yes  No 

Yes 

No 

Precision  (  )  (  ) 

Requirements 

(  ) 

(  ) 

(  )  (  ) 

(  ) 

(  ) 

Name  or  ID  of 
Output  1 

Name  or  ID  of 
Output  2 

Name  or  ID  of 
Output  3 

Name  or 
Output 

ID  of 
4# 

Output 

Yes  No 

Yes 

No 

Yes  No 

Yes 

No 

Accuracy  (  )  (  ) 

Requirements 

(  ) 

(  ) 

(  )  (  ) 

(  ) 

(  ) 

Frequency 

(  )  (  ) 

(  ) 

(  ) 

(  )  (  ) 

(  ) 

(  ) 

3.3. N  Special  Requirements .  NOTE:  The  next  seauential  number  following  the 
number  or  the  last  function. 

Yes  No 

(39)  Have  standards  to  be  followed  in  the  design  and  (  )  (  ) 

implementation  of  CPCI  computer  program  components 

(CPCs)  been  specified?  (60.5-3 *2.2) 

(40)  Are  the  standards  consistent  with  the  higher  level  (  )  (  ) 

specification  or  other  contractual  requirements  for  the 

CPCI?  (60. 5. 3- 2. 2) 

(41)  Are  the  standards  appropriate  to  the  environment  and  (  )  (  ) 

requirements  of  the  CPCI?  (60-5.3.2.2) 

(42)  Are  the  standards  compatible  and  consistent  with  those  (  )  (  ) 

that  apply  to  all  interfacing  or  otherwise  related  CPCIs? 

(60. 5. 3- 2. 2) 

(43)  Have  requirements  for  the  computer  program's  organization  (  )  (  ) 

been  specified?  (60.5-3-2.2) 

(44)  If  the  CPCI  is  to  process  classified  information,  has  (  )  (  ) 

adequate  special  attention  been  given  to  that  information's 

protection?  (60. 5-3-2. 2) 

(45)  Are  all  specific  requirements  for  growth  potential  (  )  (  ) 

identified?  For  example,  is  the  possible  hardware  expansion 

or  augmentation  for  which  the  programs  must  be  designed 
specified?  (60.5.3*2.2) 

(46)  Has  enough  performance  degradation  due  to  program  (  )  (  ) 

modification  after  initial  development  been  allowed? 

(60.5.3 -2.2) 

(47)  Have  adequate  special  features  to  facilitate  testing  been  (  )  (  ) 

specified?  (60.5-3-2.2) 


<f  Repeat  this  table  on  backup  sheets  for  any  additional  outputs. 
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Yes  No 


3.3.1  Programming  Method (s) 

(98)  Are  programming  standards  or  methods  compatible  among  (  )  (  ) 

computer  program  components?  (60.5-3-3.1) 

3.3.2  Program  Organization 

(49)  Has  special  attention  been  given  to  the  reouirements  for  (  )  (  ) 

protecting  classified  information?  (60. 5*3. 3*2) 

3.3.3  Modification  Consideration 

(50)  Have  program  design  modification  reouirements  been  (  )  (  ) 

specified?  (60.5. 3-3*3) 

3.3.4  Special  Features 

(51)  Have  aoy  special  features  or  procedures  to  facilitate  (  )  (  ) 

testing  of  the  CPCI  been  specified?  (60-5-3 .3-^*) 

3.3.5  Expandability 

(52)  Has  expandability  of  the  CPCI  been  described?  (  )  (  ) 

(60.5.3-3-5) 


3.3.6  Special  Timing 

(53)  Has  a  description  of  special  timing  on  priority  (  )  (  ) 

requirements  been  included?  (60.5.3*3.6) 

3.4  Human  Performance 


(54)  Have  all  appropriate  paragraph(s)  of  the  system  or  (  )  (  ) 

system  segment  specification  been  cited  which  establish 

the  human  per forma nee /human  engineering  requirements  for 
all  system  equipment?  (60.5.3.**) 

(55)  Have  the  human  per  forma nce/human  engineering  (  )  (  ) 

requirements  pecuLiar  to  this  CPCI  also  been  specified? 

(60. 5. 3- 4) 

(56)  Have  all  inputs  to  the  CPCI  from  a  persoo  and  outputs  (  )  (  ) 

from  the  CPCI  to  a  persoo  been  identified?  (60.5*3.4) 


(57)  For  each  such  human  input  to  the  CPCI,  has  each  of  the 
following  beeo  reasonably  specified,  considering  the  CPCI's 
and  the  system's  stated  operational  requirements?  (60.5.3.4) 


(  )  (  ) 
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Name  or 

ID  of 

Name  or 

ID 

of 

Name  or  : 

ID  of 

Name  or 

ID  of 

Output  1 

Output 

2  __ 

Output  3 

Output 

9# _ 

Human  Yes 

No 

Yes 

No 

Yes  No 

Yes 

No 

Output 

Specifications 

Content  (  ) 

(  ) 

(  ) 

( 

) 

(  ) 

(  ) 

(  ) 

(  ) 

Name  or 

ID  of 

Name  or 

ID 

of 

Name  or  ID  of 

Name  or 

ID  of 

Output  1 

— 

Output 

2  __ 

Output  3 

Output 

Hit _ 

Name  or  ID  (  ) 

(  ) 

(  ) 

( 

) 

(  ) 

(  ) 

(  ) 

(  ) 

of  Each  ttlt  Possible 

CPCI  Response 

Conditions  (  ) 

(  ) 

(  ) 

( 

) 

(  ) 

(  ) 

(  ) 

(  ) 

that  Yield  Each  #// 

Response 

Maximum  CPCI  (  ) 

(  ) 

(  ) 

( 

) 

(  ) 

(  ) 

(  ) 

(  ) 

Time  for  Each  Response 

Yes 

No 

(58)  For  each 

CPCI  output  to  a 

human, 

has  each  of 

the 

(  ) 

(  ) 

following  been 

reasonable  specified 

,  considering  the  CPCIs 

and  the  system 

'  s  stated 

operational 

reouirements? 

(60.5.3 

.9) 

Name  or 

ID  of 

Name  or 

ID 

of 

Name  or  ID  of 

Name  or 

ID  of 

Output  1 

Output 

2  __ 

Output  3 

Output 

9# _ 

CPCI  Output  Yes 

No 

Yes 

No 

Yes 

No 

Yes 

No 

Specifications 

Content  (  ) 

(  ) 

(  ) 

( 

) 

(  ) 

(  ) 

(  ) 

(  ) 

Format  (  ) 

(  ) 

(  ) 

( 

) 

(  ) 

(  ) 

(  ) 

(  ) 

Maximum  (  ) 

(  ) 

(  ) 

( 

) 

(  ) 

(  ) 

(  ) 

(  ) 

Information  Density 

Clarity  (  ) 

(  ) 

(  ) 

( 

) 

(  ) 

(  ) 

(  ) 

(  ) 

(e.g.,  Contrast  &  Brightness) 

Name  or  ID  (  ) 

(  ) 

(  ) 

( 

) 

(  ) 

(  ) 

(  ) 

(  ) 

of  Each  ittt  Corresponding  Input 

Conditions  (  ) 

(  ) 

(  ) 

( 

) 

(  ) 

(  ) 

(  ) 

(  ) 

Tnat  Yield  Each##  Input 


tt  Repeat  this  table  on  backup  sheets  for  any  additional  outputs. 

lilt  There  may  be  more  than  one  human  input/response  for  any  particular  CPCI  output. 
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Yes 


No 


3 . 5  Data  Base  Requirements 


(59)  Have  ail  specified  data  elements  been  precisely 
defined?  (60-5-3.5) 


(  ) 


i  ) 


(60)  Will  data  definitions  be  organized  by  the  system  data 
base  in  categories  meaningful  and  appropriate  to  the  given 
CPCI?  (60-5-3-5) 


(  ) 


(  ) 


(61)  Do  subparagraphs  specify  the  source  destinations  and 
types  of  input  and  output  information?  (60-5-3-5) 


(62)  Have  any  required  internal  tables  or  parameters  been 
specified?  (60-5-3-5-3) 


(  ) 

(  ) 


(  ) 


(  ) 


3-6  Adaptation 


(63)  Have  all  the  data  items  that  are  used  by  the  CPCI  and 
that  vary  from  site  to  site  been  identified  and  specified? 
(60-5-3-6) 


(  )  (  ) 


(64)  Have  all  tne  system  parameters  that  control  various 
portions  of  the  CPCI  and  that  oust  be  readily  changeable 
been  specified?  (60-5-3-6) 


(  )  (  ) 


(65)  Have  tne  CPCI's  simultaneous  capacity  requirements 
(e.g.  ,  storage,  subsystem  timing  rates,  interfacing  equipment 
capacities,  and  communication  of  data  about  the  operational 
entities  (e.g.,  total  simultaneous  displays  and  operator 
requests,  number  and  types  of  inputs  processed,  etc.))  been 
specified  quantitatively?  (60-5-3-6) 


(  )  (  ) 


Name  or  ID  of 
Output  1  _ 


Name  or  ID  of 
Output  2  _ 


Name  or  ID  of 
Output  3  _ 


Name  or  ID  of 
Output  4#  _ 


Yes 

(  ) 


No 

(  ) 


Name  or  ID 
of  Each 

Appropriate  Human  Response 


Yes 

(  ) 


No 
(  ) 


Yes 

(  ) 


No 
(  ) 


Yes 

(  ) 


No 
(  ) 


Maximum 
Time  Allowed 
A  Human  to 
Respond 


(  )  (  ) 


(  )  (  ) 


(  )  (  ) 


(  )  (  ) 


3-7  Government-Furnished  Property  List 


(66)  Have  all  Government-furnished  computer  programs,  if 
any,  that  are  to  be  iocluded  in  tne  CPCI  (including  any 
compilers  or  assemblers)  been  identified  by  proper 
nomenclature,  specification  number,  model  number ,  if 
appropriate,  aod  associated  documentation?  (60-5-3-7) 


(  )  (  ) 


It  Repeat  this  table  on  backup  sheets  for  any  additional  outputs. 

##  There  may  be  more  than  one  human  input/response  for  any  particular  CPCI  output. 
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4.  gli  ALITY  ASSURANCE  PROVISIONS 


Yes  No 


Lb 7)  Are  tne  provisions  of  this  section  consistent  with  tne  (  )  (  ) 

portions  of  the  system  specification  that  address  software 
quality  assurance? 

tod)  is  tnere  a  concise  and  unambiguous  specification  of  the  (  )  i  > 

verification  requirement  and  method(s)  (i.e.,  inspection, 

review  of  analytical  data,  demonstration,  and  review  of  text 

data)  to  be  applied  against  each  functional  and  performance 

requirement  of  Section  3?  (60.5.4  and  60.5.4.1) 

(69)  Is  there  sufficient  guidance  contained  in  this  (  )  (  ) 

specification  (or  in  referenced  higher  level  specification 
paragraphs)  for  the  preparation  of  test  plans  and  procedures 
to  insure  that  tne  CPCI  is  adequately  and  appropriately 
verified  within  reasonable  cost  and  schedule  constraints? 

(60.5.4.1) 


(70)  Are  Development  Test  &  Evaluation  (DT&E)  tests  (  )  (  ) 

subdivided  into  following  broad  types:  (A)  Computer 
Programming  Test  and  Evaluation  (CPT&E);  (b)  Preliminary 
Qualification  Tests  (PQT);  (c)  Formal  Qualification  Tests 
(FQT)?  (60.5.4.1) 

(7D  Do  the  tests  compromising  CPT&E  satisfy  one  or  both  (  )  (  ) 

of  the  following  criteria?  (60.5.4.1) 


Name  or  ID  of 

Name  or 

ID  of 

Name  or  ID  of 

Output  1 

Output 

2 

Output  3 

Criteria  Yes  No 

Yes 

No 

Yes  No 

Only  data  (  )  (  ) 

source 

qua lifyiog 

Section  3 

Requirements 

(  ) 

(  ) 

(  )  (  ) 

Accomplished  (  )  (  ) 

as  part  of 

(  ) 

(  ) 

(  )  (  ) 

integrated  test 
program  involving 
other  systems, 
equipment,  or 
computer  programs? 

(72)  Are  any  applicable  special  haodling  requirements 
described?  (60.5.5) 


Name  or  ID  of 
Output  4#  _ 

Yes  No 

(  )  (  ) 


(  )  (  ) 


(  )  (  ) 


#  Repeat  this  table  on  backup  sheets  for  any  additional  outputs. 
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Section  9  Computer  Program  Product  Specification 


9-1.  He  view  draft  C-6  Spec  for  tecnnical  adequacy  and  satisfaction  of  each 
requirement  in  the  respective  B-5  specification. 

9-2.  Heview  MIL-STD-483,  Appendix  VI,  Computer  Program  Configuration  Item 
Specification  to  insure  the  C-5  is  in  compliance. 

9-3.  Heview  ESD-TR-77-22 ,  Life  Cycle  Eveots,  Section  A4,  Computer  Program  Product 
Specifications,  for  compliance  requirements. 

9-4.  Coordinate  review  of  tne  document  with  other  Government  ageocies  involved. 

9-5-  Consolidate  comments  from  other  Government  agencies  and  forward  to  the 
contractor . 

9-6.  Review  all  changes  to  the  document  as  a  result  of  Government  comments. 


9-7.  Coordinate  all  changes  witn  all  agencies  involved. 

9-8.  Heview  the  Computer  Program  Product  Specification  Checklist,  Figure  III-9-1 
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Figure  III-9-1  COMPUTER  PROGRAM  PRODUCT  SPECIFICATION  CHECKLIST 
(CPCI  Part  II  Specification  I  AW  Notice  2  to  MI L-STD-483 ) 

This  cnecklist  is  not  intended  to  be  a  substitute  for  thorough  understanding  of  the 
overaLl  system  requirements  and  design  approach.  Familiarity  with  the 
corresponding  development  specification,  related  CPCI  and  Cl  specifications  and 
ni^ner  specif ications  is  considered  a  prerequisite  for  a  meaningful  review. 

Tne  folLowiog  list  is  based  on  the  requirements  of  MIL-STD-483  (USAF)  ,  Appendix 
VI.  Tne  reviewer  must  be  aware  of  all  applicable  standards  plus  any  supplemental 
direction  supplied  in  the  SOW  or  CDRL  backup  sheets.  Such  direction  may  be  used  to 
modify  this  cnecKList  in  a  specific  application.  Section  numbers  prescribed  in 
MIL-STD-483,  Appendix  VI,  are  shown  ir,  parentheses  following  the  questions.  An 
in  a  section  oumber  indicates  a  section  repeated  for  each  CPC,  so  X  is  1  tor  the 
first  CPC,  2  for  the  second,  etc. 

1.  SCOPE  Yes  No 

(1)  Does  this  section  contain  the  appropriate  lead  sentence?  (  )  (  ) 

(60.5.1) 

2.  APPLICABLE  DOCUMENTS 

l 2 )  Are  all  documents  properly  listed?  (60.6.2)  (  )  (  ) 

3.  REQUIREMENTS 

3.1  CPCI  Structured  Description 

(3)  Does  tne  specification  identify  tne  structure  and  (  )  (  ) 

functions  of  the  CPCI  consistent  with  the  development 

specification  (including  approved  changes)?  (60. 6. 3.1) 

(4)  Does  the  design  reflect  the  results  or  previous  action  (  )  (  ) 

items  identified  in  earlier  design  reviews?  (60.6-3- l ) 

(5)  Does  this  section  include  a  graphic  portrayal  (CPCI/CPC  (  )  (  ) 

tree)?  (6O.6.3.I) 

3 . 2  Functional  Flow  Diagrams/Charts 

(6)  Is  the  general  system  flow  of  both  data  and  control  (  )  (  ) 

within  the  CPCI  shown?  (60.6.3-2) 

(7)  Are  the  operations  performed  by  the  CPCI  graphically  (  )  (  ) 

portrayed  by  a  (series  of)  chart! s)  which  depict(s)  the 

processing  being  performed? 


(8)  Does  the  CPC  description  and  flowcharts  provide 
appropriate  level  of  detail  for  modifying  and  maintaining 
the  source  program  (listing)  consistent  with  the  intended 
usage  of  the  product  specification  and  any  specific  contract 
requirements?  (60.6.3.2) 
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8 •  6  liming  and  Sequencing  Description 

(181  Does  a  description  ot  tne  control  logic  involved  in 
referencing  each  CPC  exist?  (60.6. 3. 5) 

(lyi  Are  timing  and  sequencing  of  CPC  operations  relative  to 
otner  CPCs  described  or  pictorially  depicted?  (60.6.3.5) 

(20)  Are  tne  (critical)  execution  timing  estimates  or 
measurements  provided?  Do  they  conform  to  the 
requirements  of  the  Development  Spec?  (60.6.3.5) 

(PI)  Is  the  method  for  computing  spare  time  and  the  amount 
allocated  as  spare  time  described?  (60.6. 3.5) 


Ye3  No 


(9)  Is  a  top  level  CPC  flowchart  provided  that  relates  the 
processes  being  performed,  the  sequence  of  operating  and 
decision  points? 


(  )  (  ) 


(10)  Do  tne  flowcharts  reference  eacn  otner  properly?  Are 
the  flowcharts  referenced  to  actual  statement  labels 

or  tags  in  tne  listings  (when  available)?  (60.6.3*2) 

(11)  Do  flowcharts  correspond  to  the  task  descriptions? 
(60.O.3.2) 


(  )  (  ) 


(  )  (  ) 


(12)  li  flowchart  symboLogy  in  accordance  with  the  current 
edition  of  Flowchart  Symbols  and  Their  Usage  in 
Information  Processing,  ANSI  X3-5-1970  or  an  approved 
standard?  (60.6.3*2) 


(  )  (  ) 


(13)  Are  CPC1  mooes  of  operation  clearly  distinguishable  in  (  )  (  ) 

both  text  aod  charts,  (if  applicable)?  (60.6.3*2) 


3.  j  Interlaces 

(14)  ii  tne  essential  as-built  interface  design  between  the 
CPCI  and  other  operational  computer  programs  and  hardware 
described  in  detail?  (60.6. 3. 3) 


(  )  (  ) 


(15)  Are  all  data  units  logically  and  quantitatively 
described?  (60. 6. 3* 3) 


(  )  (  ) 


3-4  Program  Interrupts 

(16)  Does  tne  program  interrupt  list  describe  each  interrupt  (  )  (  ) 

as  to  source,  purpose,  type,  priority  and  required 

response  of  the  executive  control?  (60.6.3.4) 

(17)  Are  interrupt  servicing  components  listed  along  with  (  )  (  ) 

tne  proDable  rate  of  occurrence  for  each  interrupt? 

(60.6. 3.4) 
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j.6  Special  Control  Features  Yes  No 

( 22 )  Are  the  special  control  features  outside  of  normal  (  )  (  ) 

operational  functions  (such  as  initiation,  restart, 

recovery,  maintenance)  identified  and  described? 

3-7  Storage  Allocation 

(23)  Are  CPCI  storage  requirements  in  relationship  to  total  (  )  (  ) 

computer  equipment  storage  capabilities  graphically 

portrayed?  (60.6.3.7) 

3.7- 1  Data  Base  Definition 

(24)  Does  a  description  of  each  file,  table  or  item  as  to  (  )  (  ) 

functional  purpose,  detailed  definition  of  cootent  and 

storage  locatioo  exist  for  the  CPCI  data  base? 

(60. 6. 3-7-1) 

3-7. 1.1  File  Description 

(25)  Has  each  file  incorporated  in  the  data  base  been  (  )  (  ) 

described?  (60. 6-3.7- 1. 1) 

3 . 7 -  1 • 2  Table  Description 

(26)  Has  each  table  incorporated  in  the  CP  data  base  been  (  )  (  ) 

described?  (60. 6. 3*7- 1.2) 

3-7. 1.3  Item  Description 

(27)  Are  all  items  contained  in  the  computer  program  data  (  )  (  ) 

base  described?  (60.6.3«7.l«3) 

3-7. 1.4  Graphic  Table  Description 

(28)  Does  the  description  show  all  interrelationships  (  )  (  ) 

between  items,  tables  and  files?  (60.6.3.7-1.4) 

(29)  Is  the  table  accomplished  to  the  level  of  detail  (  )  (  ) 

necessary  to  show  required  items?  (60.6.3.7.1.4) 

3.7. 1.5  CPCI  Constants 

(30)  Are  the  actual  data  values  for  all  constants  (  )  (  ) 

defined?  (60.6.3*7.1.5) 

3.7.2  CPC  Relationship 

(3D  Is  the  relationship  between  tne  CPC  aod  the  data  base  (  )  (  ) 

given?  (60.6.3*7.2) 
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Yes  No 


3.7.3  Data  Base  Location  Requirements 

(3 2)  Are  storage  location  requirements  described,  if  (  )  ( 

applicable?  (60.6.3-7-3) 

3.8  Object  Code  Creation 

(33)  Are  all  object  code  generation  requirements  identified?  (  )  ( 

(60.6-3-8) 

3.9  Adaption  Data 

(34)  Ii  tne  adaption  data  specified  for  each  unique  site/  (  )  ( 

configuration?  Is  it  consistent  with  the  System 

Allocation  Document  (if  used)?  (6O.6.3.9) 

3.10  Detail  Design  Description 

(36)  Are  individual  CPC*  described  at  the  level  of  detail  (  )  ( 

that  define  the  design  to  permit  CPC  modification  and 

adaption  during  operational  use  of  the  CPCI?  Are  they 

described  with  words  and  charts  with  necessary  inter- 

reierences  for  clarity?  (60.6. 3. 10) 

(36)  Does  the  lead  paragraph  follow  the  required  format?  (  )  ( 

(60. 6. 3. 10) 

3-10.X**  Identification 

(37)  Is  eacn  CPC  identified  by  a  name,  tag  and  (  )  ( 

identification  number?  (60. 6. 3.10.1) 

3-10.X.1  CPC  No.  X  Description 

(36)  Are  conventions  for  symbol  naming,  register  usage  (  )  ( 

and  coding  described?  (60. 6. 3. 10. 1.1) 

(39)  Are  the  CPC  task  descriptions  consistent  with  (  )  ( 

tne  requirements  of  the  development  specification 

and  allocation  of  functions  to  the  CPC? 

(60.6.3.)0.1.l) 

(40)  Is  tne  hierarchy  chart  used  to  show  any  (  )  ( 

intrarelationships?  (60. 6. 3. 10. 1.1) 


••3.IO.X. 1  thru  3.I0.X. 6  is  repeated  for  each  CPC 
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3-10. X. 2  CPC  No.  X  Cnarts  Yes  No 

(41)  Do  the  flowcharts  reference  each  other  properly?  (  )  (  ) 

Are  the  flowcharts  referenced  to  actual  statement 

labels  or  tags  in  the  listiogs  (when  available)? 

Do  tne  flow  charts  correspond  to  the  task  descriptions? 

(60.6.3.10.1.2) 

3.10. X. 3  CPC  No.  X  Interfaces 

(42)  Are  tne  interfaces  to  other  CPCs  reflected  in  the  (  )  (  ) 

descriptions?  Is  the  same  terminology  used?  Are 

interfaces  to  the  data  base  (external  to  the  CPC) 
identified  and  described?  Are  those  interfaces 
consistent  with  tne  data  base  description?  (See  Section 
3-7.1). 

(43)  Are  the  interfaces  to  other  CPCIs  or  CIs  (  )  (  ) 

identified  and  described?  Are  the  I/O  formats 

and  butter  definitions  provided  or  correctly 
referenced?  Do  the  external  interfaces  correspond 
to  tne  related  interlaces  ana  descriptions  on  those 
other  (CP)  Cl  specifications?  Do  they  reflect  changes 
Implemented  on  those  other  (CP)  Cl  specifications? 

3.10. X. 4  CPC  No.  X  Data  Organization 

(44)  Are  the  symbols,  constants,  equivalences,  data  (  )  (  ) 

items  and  tables  unique  to  the  CPC  described? 

(60.6.3.10.1.4) 

(46)  la  the  usage  of  temporary  (dynamic)  storage  areas  (  )  (  ) 

described?  (60.6.3- 10.1.4) 

(46)  Are  the  relationships  among  the  CPCI  data  items,  (  )  (  ) 

tables  aod  files  defined?  (60.6.3.10.1.4) 

(47)  Are  tne  storage  allocations  (starting  address  and  (  )  (  ) 

extent)  of  all  data  structures  aod  CPCs  of  the 

CPCI  specified?  (60.6.3*10.1.4) 

(48)  Are  the  files  (unique  to  the  CPCI)  described  in  (  )  (  ) 

terms  of  length,  format/oaming  convention, 

organization,  access  method  blocking,  buffering? 

Are  the  tables  similarly  defined?  Are  the  data 
items  described  in  terms  of  number  of  bits,  most 
significant  bit,  coding  type,  scaling  factor,  units, 
range  of  values,  etc?  (60.6.3*10.1.4) 

(49)  Is  a  set-use  matrix  (or  equivalent)  provided  that  (  )  (  ) 

details  use  of  all  data  items,  tables,  files, 

constants,  etc.,  by  each  CPC?  (60.6.3.10.1.4; 


29 


HELPL,  Volume  III 

Kevi.ewu,  Audits,  ana  CPCI  Specif icatioos 


1  February  19&3 


3 . 1 0 .  X .  6  CPC  No.  Limitations 

(SO)  Are  the  limitations  of  the  implementation  described 
with  respect  to  timing  requirements,  algorithms 
employed,  I/O  t'ormats,  I/O  error  checking  and  correcting 
and  accuracy?  (60 . 6 . 3 • 10. 1 .5 ) 

1.10. X.6  CPC  No.  X  Listing 

(‘>1)  Does  tne  listing  (wnen  available)  conform  to  the 
contractual  requirements  for  source  listing,  compiler/ 
assembler  output,  cross  reference  listing,  etc.?  Does  the 
listing  include  the  statement  labels,  tags  and  comments 
referenced  in  the  flowcharts?  Does  the  listing  conform  to 
the  task  descriptions  and  flowcharts?  ( 10 .6.3* 10. I .6) 

3 .  L  L  Program  Listing  Comments 

(6.D  Do  program  listings  comments  meet  contractual  (  )  (  ) 

requirements  as  to  type,  format  and  level  of  detail? 

(60.6.3*  ID 

4.  QUALITY  ASSURANCE 

(63)  Are  tne  associated  CPCI  test  plan  and  procedures  (  )  (  ) 

identified?  (60.6.4) 

4 . 1  Te3t  Plan/Procedure  Cross  Reference  Iodex 

(64)  Are  all  the  CPCI  functions  cross  referenced  to  the  (  )  (  ) 

applicable  test  plan  and  procedures?  (60.6.4.1) 

4 . d  Other  Quality  Assurance  Provisions 

(66)  Are  special  test  tools  or  other  testing  capabilities  (  )  (  ) 

used  in  CPCI  level  testing  identified?  (60.6.4.2) 

(66)  Are  quality  assurance  requirements,  methods  and  (  )  (  ) 

procedures  for  preparation  or  duplication  of  the 

program  (media)  either  specified  or  the  applicable 
standards  referenced?  (60.6.4.2) 

(67)  If  there  is  particular  information  important  to  the  (  )  (  ) 

use  of  the  specification  or  computer  program  in  the 

future  (e.g.,  integration  testing),  is  it  provided? 

(60.6.4.2) 


Yes  No 
(  )  (  ) 


(  )  (  ) 


PREPARATION  FOR  DELIVERY 


I 


,  \. 


r  •  * 


LtS 


>> 


6. 1  ackagj.ng 

(66)  Are  tne  requirements  lor  packaging  specified  and  (  )  (  ) 

consistent  with  anticipated  handling  and  storage?  (60.6.5.1) 
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5.2  Markings 

(69)  Are  the  iaentif ication  markings  detailed  and  consistent 
with  the  requirements?  (60. 6. 5*2) 

6.  NOTES 

(60)  Are  the  tecnnical  manuals  necessary  for  proper  CPCI 
operation  and  maintenance  identified?  (60.6.6) 

(bl)  Is  additional  material  not  included  elsewhere  provided 
to  support  particular  CPCs,  such  as  alternative  designs 
rejected,  design  rationale,  references  for  algorithms  used, 
suggestions  for  further  modifications  or  expansion,  results 
of  key  test  that  finalized  the  implementation?  (60.6.6) 

7.  SECTION  10,  APPENDIX! ES) 

(62)  Is  information  as  referenced  from  other  sections  provided 
in  the  Appendix?  (60.6.7) 


Yes  No 
(  )  ( 

(  )  ( 

(  )  ( 


(  )  ( 
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